Coordinating Momenta for Cross-Silo Federated Learning
نویسندگان
چکیده
Communication efficiency is crucial for federated learning (FL). Conducting local training steps in clients to reduce the communication frequency between and server a common method address this issue. However, it leads client drift problem due non-i.i.d. data distributions different which severely deteriorates performance. In work, we propose new improve performance cross-silo FL via maintaining double momentum buffers. One buffer tracks model updating direction, other direction. Moreover, introduce novel fusion technique coordinate We also provide first theoretical convergence analysis involving both standard SGD. Extensive deep experimental results show better than FedAvg existing SGD variants.
منابع مشابه
Coordinating Policy for Federated Applications
At the start of its present term of office in 1997 the UK government published a planning document promising ubiquitous access to Electronic Health Records (EHRs) held within the National Health Service (NHS). If such access is to become a reality then it is essential to guarantee confidentiality, since otherwise the media and the privacy vigilantes will prevent deployment. Among the rights inc...
متن کاملFederated Meta-Learning for Recommendation
Recommender systems have been widely studied from the machine learning perspective, where it is crucial to share information among users while preserving user privacy. In this work, we present a federated meta-learning framework for recommendation in which user information is shared at the level of algorithm, instead of model or data adopted in previous approaches. In this framework, user-speci...
متن کاملFederated Multi-Task Learning
Federated learning poses new statistical and systems challenges in training machinelearning models over distributed networks of devices. In this work, we show thatmulti-task learning is naturally suited to handle the statistical challenges of thissetting, and propose a novel systems-aware optimization method, MOCHA, that isrobust to practical systems issues. Our method and theor...
متن کاملFederated Learning: Strategies for Improving Communication Efficiency
Federated Learning is a machine learning setting where the goal is to train a highquality centralized model with training data distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2022
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v36i8.20853